EDA on New TSX List

Goals

• Analyse composition of stocks from TSX Issuers related to Construction and Real Estate

Notes

• Data directly from TSX at https://www.tsx.com/listings/current-market-statistics and specifically. This data we would want to ingest at some point.

rm(list=ls())

library("readxl")
library(quantmod)
library(dplyr)
library(ggplot2)

I. Loading and Preprocessing of Excel file

The file below was downloaded 26/10/2020 from the TSX website above. File contains first some rows with non-relevant information to skip

issuers <- read_excel(paste("../../../data_tmp/tsx-andamp-tsxv-listed", 
                            "-companies-2020-10-16-en-1.xlsx", sep=""), 
                      sheet=2, skip=6)

-
/
                                                                                                                                          

head(issuers, 2)

colnames(issuers) <- lapply(str_replace_all(colnames(issuers), "[\r\n -$\\(\\)\\-]", "_"), FUN=tolower)

Discard companies from the sector Real Estate Operating Companies as not relevant based on Elton’s comments.

issuers <- issuers %>% filter(real_estate_sub_sector != 
                      "Real Estate Operating Companies")

real_estate <- filter(issuers, !is.na(issuers[, 'real_estate_sub_sector']))[,  c("real_estate_sub_sector", "name", "root__ticker", "market_cap__c____30_september_2020")]
head(real_estate)

Download of Stocks based on Tickers

Root ticker are not representative of the TSX

head(real_estate)  

Use ticker column to download data. Try statement to catch errors.

#stock_name <- "BPY-UN.TO"
get_stockdata <- function(stock_name_root) {
  out <- try ({
    stock_name <- paste(stock_name_root, "-UN.TO", sep="")
    stock <- getSymbols(stock_name, src = "yahoo", auto.assign = FALSE) 
    # calculate monthly returns instead of using all stock data
    stock = monthlyReturn(stock)
    colnames(stock) <- sapply(colnames(stock), function (colstock) str_split(colstock, "\\.")[[1]][length(str_split(colstock, "\\.")[[1]])])
    stock <- data.frame(date=index(stock), coredata(stock))
    stock$stock_name <- rep(stock_name, nrow(stock))
    stock$stock_name_root <- rep(stock_name_root, nrow(stock))
    stock
  })
  
  # if stock is not available as "[stockname]-UN.TO", try "[stockname"].TO"
  if (class(out) == "try-error") {
  
    out <- try ({
      stock_name <- paste(stock_name_root, ".TO", sep="")
      stock <- getSymbols(stock_name, src = "yahoo", auto.assign = FALSE) 
    # calculate monthly returns instead of using all stock data
      stock = monthlyReturn(stock)
      colnames(stock) <- sapply(colnames(stock), function (colstock) str_split(colstock, "\\.")[[1]][length(str_split(colstock, "\\.")[[1]])])
      stock <- data.frame(date=index(stock), coredata(stock))
      stock$stock_name <- rep(stock_name, nrow(stock))
      stock$stock_name_root <- rep(stock_name_root, nrow(stock))
      stock
    })
  }
  
  out
  
  }

start.time <- Sys.time()
get_stock_series <- lapply(real_estate$root__ticker, get_stockdata)

AP-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataHOT-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataAX-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataAPR-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataBEI-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataBRE-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "BRE-UN.TO", env = <environment>,  : 
  Unable to import “BRE-UN.TO”.
BRE-UN.TO download failed after two attempts. Error message:
HTTP error 404.
BRE.TO download failed; trying again.Warning in for (i in seq_len(n)) { :
  closing unused connection 5 (https://query1.finance.yahoo.com/v7/finance/download/BRE.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=nl4VMJD9Hc5)
Warning in for (i in seq_len(n)) { :
  closing unused connection 4 (https://query1.finance.yahoo.com/v7/finance/download/BRE-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=nl4VMJD9Hc5)
Warning in for (i in seq_len(n)) { :
  closing unused connection 3 (https://query1.finance.yahoo.com/v7/finance/download/BRE-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=NcIouEhK9yv)
BPO-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "BPO-UN.TO", env = <environment>,  : 
  Unable to import “BPO-UN.TO”.
BPO-UN.TO download failed after two attempts. Error message:
HTTP error 404.
BPO.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "BPO.TO", env = <environment>, verbose = FALSE,  : 
  Unable to import “BPO.TO”.
BPO.TO download failed after two attempts. Error message:
HTTP error 404.
BPY-UN.TO download failed; trying again.BPY-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataHOM-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataBTB-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataCAR-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataCSH-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataCHP-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataCUF-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataCRR-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataCRT-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataDRA-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataDIR-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataD-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataDRM-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "DRM-UN.TO", env = <environment>,  : 
  Unable to import “DRM-UN.TO”.
DRM-UN.TO download failed after two attempts. Error message:
HTTP error 404.
DRM.TO download failed; trying again.ERE-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "ERE-UN.TO", env = <environment>,  : 
  Unable to import “ERE-UN.TO”.
ERE-UN.TO download failed after two attempts. Error message:
HTTP error 404.
ERE.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "ERE.TO", env = <environment>, verbose = FALSE,  : 
  Unable to import “ERE.TO”.
ERE.TO download failed after two attempts. Error message:
HTTP error 404.
EXE-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "EXE-UN.TO", env = <environment>,  : 
  Unable to import “EXE-UN.TO”.
EXE-UN.TO download failed after two attempts. Error message:
HTTP error 404.
EXE.TO download failed; trying again.FCR-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataFSV-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "FSV-UN.TO", env = <environment>,  : 
  Unable to import “FSV-UN.TO”.
FSV-UN.TO download failed after two attempts. Error message:
HTTP error 404.
FSV.TO download failed; trying again.GDC-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "GDC-UN.TO", env = <environment>,  : 
  Unable to import “GDC-UN.TO”.
GDC-UN.TO download failed after two attempts. Error message:
HTTP error 404.
GDC.TO download failed; trying again.HR-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataINO-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataIIP-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataIVQ-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "IVQ-UN.TO", env = <environment>,  : 
  Unable to import “IVQ-UN.TO”.
IVQ-UN.TO download failed after two attempts. Error message:
HTTP error 404.
IVQ.TO download failed; trying again.DR-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "DR-UN.TO", env = <environment>, verbose = FALSE,  : 
  Unable to import “DR-UN.TO”.
DR-UN.TO download failed after two attempts. Error message:
HTTP error 404.
DR.TO download failed; trying again.MRD-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "MRD-UN.TO", env = <environment>,  : 
  Unable to import “MRD-UN.TO”.
MRD-UN.TO download failed after two attempts. Error message:
HTTP error 404.
MRD.TO download failed; trying again.MR-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataMI-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataMRC-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "MRC-UN.TO", env = <environment>,  : 
  Unable to import “MRC-UN.TO”.
MRC-UN.TO download failed after two attempts. Error message:
HTTP error 404.
MRC.TO download failed; trying again.MRG-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataMRT-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataNVU-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataNWH-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataPLZ-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataPRV-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataREI-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataSIA-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "SIA-UN.TO", env = <environment>,  : 
  Unable to import “SIA-UN.TO”.
SIA-UN.TO download failed after two attempts. Error message:
HTTP error 404.
SIA.TO download failed; trying again.SIA.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataSGR-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "SGR-UN.TO", env = <environment>,  : 
  Unable to import “SGR-UN.TO”.
SGR-UN.TO download failed after two attempts. Error message:
HTTP error 404.
SGR.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "SGR.TO", env = <environment>, verbose = FALSE,  : 
  Unable to import “SGR.TO”.
SGR.TO download failed after two attempts. Error message:
HTTP error 404.
SOT-UN.TO download failed; trying again.SOT-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataSRU-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataSMU-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataTNT-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataWFC-UN.TO download failed; trying again.Error in getSymbols.yahoo(Symbols = "WFC-UN.TO", env = <environment>,  : 
  Unable to import “WFC-UN.TO”.
WFC-UN.TO download failed after two attempts. Error message:
HTTP error 404.
WFC.TO download failed; trying again.WIR-UN.TO contains missing values. Some functions will not work if objects contain missing values in the middle of the series. Consider using na.omit(), na.approx(), na.fill(), etc to remove or replace them.missing values removed from dataWarning in give.length :
  closing unused connection 46 (https://query2.finance.yahoo.com/v7/finance/download/WFC.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=M5w7sOsNvPB)
Warning in give.length :
  closing unused connection 45 (https://query2.finance.yahoo.com/v7/finance/download/WFC-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=M5w7sOsNvPB)
Warning in give.length :
  closing unused connection 44 (https://query2.finance.yahoo.com/v7/finance/download/WFC-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=6KRUV9eJGJG)
Warning in give.length :
  closing unused connection 43 (https://query1.finance.yahoo.com/v7/finance/download/SOT-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=pmLIa4.gDQY)
Warning in give.length :
  closing unused connection 42 (https://query1.finance.yahoo.com/v7/finance/download/SGR.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=pmLIa4.gDQY)
Warning in give.length :
  closing unused connection 41 (https://query2.finance.yahoo.com/v7/finance/download/SGR.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=Uk9oVSWRrOk)
Warning in give.length :
  closing unused connection 40 (https://query2.finance.yahoo.com/v7/finance/download/SGR-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=Uk9oVSWRrOk)
Warning in give.length :
  closing unused connection 39 (https://query2.finance.yahoo.com/v7/finance/download/SGR-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=C.7/ROl4aHg)
Warning in give.length :
  closing unused connection 38 (https://query1.finance.yahoo.com/v7/finance/download/SIA.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=y0UdPNvuvZK)
Warning in give.length :
  closing unused connection 37 (https://query2.finance.yahoo.com/v7/finance/download/SIA-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=y0UdPNvuvZK)
Warning in give.length :
  closing unused connection 36 (https://query1.finance.yahoo.com/v7/finance/download/SIA-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=BHla2xJaBvD)
Warning in give.length :
  closing unused connection 35 (https://query2.finance.yahoo.com/v7/finance/download/MRC.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=87Fsk8iwI2Q)
Warning in give.length :
  closing unused connection 34 (https://query2.finance.yahoo.com/v7/finance/download/MRC-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=87Fsk8iwI2Q)
Warning in give.length :
  closing unused connection 33 (https://query2.finance.yahoo.com/v7/finance/download/MRC-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=V2vxkijwfAk)
Warning in give.length :
  closing unused connection 32 (https://query2.finance.yahoo.com/v7/finance/download/MRD.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=W1etzULAOmn)
Warning in give.length :
  closing unused connection 31 (https://query2.finance.yahoo.com/v7/finance/download/MRD-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=W1etzULAOmn)
Warning in give.length :
  closing unused connection 30 (https://query2.finance.yahoo.com/v7/finance/download/MRD-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=Ht/F4yKThEH)
Warning in give.length :
  closing unused connection 29 (https://query2.finance.yahoo.com/v7/finance/download/DR.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=SVaFsxbcFjd)
Warning in give.length :
  closing unused connection 28 (https://query2.finance.yahoo.com/v7/finance/download/DR-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=SVaFsxbcFjd)
Warning in give.length :
  closing unused connection 27 (https://query1.finance.yahoo.com/v7/finance/download/DR-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=ysu8EfYbbSB)
Warning in give.length :
  closing unused connection 26 (https://query1.finance.yahoo.com/v7/finance/download/IVQ.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=sBek06jn/nm)
Warning in give.length :
  closing unused connection 25 (https://query1.finance.yahoo.com/v7/finance/download/IVQ-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=sBek06jn/nm)
Warning in give.length :
  closing unused connection 24 (https://query2.finance.yahoo.com/v7/finance/download/IVQ-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=czs7jtF1.Hh)
Warning in give.length :
  closing unused connection 23 (https://query1.finance.yahoo.com/v7/finance/download/GDC.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=F.PmmZDRfhG)
Warning in give.length :
  closing unused connection 22 (https://query1.finance.yahoo.com/v7/finance/download/GDC-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=F.PmmZDRfhG)
Warning in give.length :
  closing unused connection 21 (https://query1.finance.yahoo.com/v7/finance/download/GDC-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=H7tQebwunW4)
Warning in give.length :
  closing unused connection 20 (https://query1.finance.yahoo.com/v7/finance/download/FSV.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=ptaiPK9aWR1)
Warning in give.length :
  closing unused connection 19 (https://query2.finance.yahoo.com/v7/finance/download/FSV-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=ptaiPK9aWR1)
Warning in give.length :
  closing unused connection 18 (https://query1.finance.yahoo.com/v7/finance/download/FSV-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=1/fy1zhye1Q)
Warning in give.length :
  closing unused connection 17 (https://query1.finance.yahoo.com/v7/finance/download/EXE.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=I/2tiCmgP7.)
Warning in give.length :
  closing unused connection 16 (https://query1.finance.yahoo.com/v7/finance/download/EXE-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=I/2tiCmgP7.)
Warning in give.length :
  closing unused connection 15 (https://query1.finance.yahoo.com/v7/finance/download/EXE-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=xIbHqGagIDx)
Warning in give.length :
  closing unused connection 14 (https://query1.finance.yahoo.com/v7/finance/download/ERE.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=xIbHqGagIDx)
Warning in give.length :
  closing unused connection 13 (https://query1.finance.yahoo.com/v7/finance/download/ERE.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=3z7V5.M3EfI)
Warning in give.length :
  closing unused connection 12 (https://query1.finance.yahoo.com/v7/finance/download/ERE-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=3z7V5.M3EfI)
Warning in give.length :
  closing unused connection 11 (https://query1.finance.yahoo.com/v7/finance/download/ERE-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=0fce15F478c)
Warning in give.length :
  closing unused connection 10 (https://query2.finance.yahoo.com/v7/finance/download/DRM.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=bYrZEcpla0V)
Warning in give.length :
  closing unused connection 9 (https://query2.finance.yahoo.com/v7/finance/download/DRM-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=bYrZEcpla0V)
Warning in give.length :
  closing unused connection 8 (https://query2.finance.yahoo.com/v7/finance/download/DRM-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=K3KhgFHo9vF)
Warning in give.length :
  closing unused connection 7 (https://query2.finance.yahoo.com/v7/finance/download/BPY-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=/y2DZjZodnj)
Warning in give.length :
  closing unused connection 6 (https://query2.finance.yahoo.com/v7/finance/download/BPO.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=/y2DZjZodnj)
Warning in give.length :
  closing unused connection 5 (https://query1.finance.yahoo.com/v7/finance/download/BPO.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=o1KlxQmzm0l)
Warning in give.length :
  closing unused connection 4 (https://query2.finance.yahoo.com/v7/finance/download/BPO-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=o1KlxQmzm0l)
Warning in give.length :
  closing unused connection 3 (https://query1.finance.yahoo.com/v7/finance/download/BPO-UN.TO?period1=1167609600&period2=1604016000&interval=1d&events=history&crumb=y3ze1kyJkU9)

print(paste("total download time :", Sys.time() - start.time, "s"))

[1] "total download time : 1.38832774957021 s"

Identify which time series was not downloaded successfully and print their error messages. Successfull stocks are saved in stock_series.

downloaded <- list()
stock_series <- list()
j = 1
k = 1
for (i in 1:length(get_stock_series)) {
  if (class(get_stock_series[[i]]) == "try-error") {
    #missing_data_log[[k]] <- get_stock_series[[i]]
    print(get_stock_series[[i]])
    downloaded[[i]] <- FALSE
  } else {
    downloaded[[i]] <- TRUE
    stock_series[[j]] <- get_stock_series[[i]]
    j <- j + 1
  }
}

[1] "Error in getSymbols.yahoo(Symbols = \"BPO.TO\", env = <environment>, verbose = FALSE,  : \n  Unable to import “BPO.TO”.\nBPO.TO download failed after two attempts. Error message:\nHTTP error 404.\n"
attr(,"class")
[1] "try-error"
attr(,"condition")
<simpleError in getSymbols.yahoo(Symbols = "BPO.TO", env = <environment>, verbose = FALSE,     warnings = TRUE, auto.assign = FALSE, .has1sym. = TRUE): Unable to import “BPO.TO”.
BPO.TO download failed after two attempts. Error message:
HTTP error 404.>
[1] "Error in getSymbols.yahoo(Symbols = \"ERE.TO\", env = <environment>, verbose = FALSE,  : \n  Unable to import “ERE.TO”.\nERE.TO download failed after two attempts. Error message:\nHTTP error 404.\n"
attr(,"class")
[1] "try-error"
attr(,"condition")
<simpleError in getSymbols.yahoo(Symbols = "ERE.TO", env = <environment>, verbose = FALSE,     warnings = TRUE, auto.assign = FALSE, .has1sym. = TRUE): Unable to import “ERE.TO”.
ERE.TO download failed after two attempts. Error message:
HTTP error 404.>
[1] "Error in getSymbols.yahoo(Symbols = \"SGR.TO\", env = <environment>, verbose = FALSE,  : \n  Unable to import “SGR.TO”.\nSGR.TO download failed after two attempts. Error message:\nHTTP error 404.\n"
attr(,"class")
[1] "try-error"
attr(,"condition")
<simpleError in getSymbols.yahoo(Symbols = "SGR.TO", env = <environment>, verbose = FALSE,     warnings = TRUE, auto.assign = FALSE, .has1sym. = TRUE): Unable to import “SGR.TO”.
SGR.TO download failed after two attempts. Error message:
HTTP error 404.>

real_estate['downloaded'] <- unlist(list(downloaded))

Analysis of missing data

A small amount of stocks were not downloaded/are not available

table(real_estate$downloaded)

FALSE  TRUE 
    3    46 

The following 3 stocks could not be downloaded. In fact * BPO.TO: shows NA values in Yahoo: https://ca.finance.yahoo.com/quote/bpo.to/ * ERE: not available on Yahoo, should be available on TSX, according to https://www.eresreit.com. As its European I discard the ticker for now. * SGR: should be availalbe at https://ca.finance.yahoo.com/quote/SGR-UN.TO/ Note that all 3 missing stocks are part of the Industrial/Office/Retail/Residential subsector for which plenty of companies are available already. Discard the companies below in further work.

real_estate[ !real_estate$downloaded, ]

II. Analysis of stocks Properties

stock_series <- bind_rows(stock_series)

head(stock_series)

add sub-sector information to stocks

stock_series <- merge(stock_series, real_estate, by.x="stock_name_root", by.y="root__ticker", all=FALSE)
head(stock_series, 2)

table(real_estate[real_estate$downloaded, ]$real_estate_sub_sector)

                         Diversified Industrial/Office/Retail/Residential Real Estate Development and Services                          Specialized 
                                   7                                   30                                    3                                    6 

1. Category: Diversified

Most companies go back to 2007 but those with the largest market cap go back only to 2012. Fair amount of distribution.

stock_series_stats_div <- stock_series %>% 
  filter(real_estate_sub_sector=="Diversified") %>%
  group_by(stock_name_root) %>%
      summarize(
        min = min(date, na.rm=TRUE),
        max = max(date, na.rm=TRUE),
        real_estate_sub_sector = unique(real_estate_sub_sector),
        market_cap__c____30_september_2020 = 
          max(market_cap__c____30_september_2020),
        name = unique(name),
        .groups = "keep"
      )
stock_series_stats_div$market_cap_fraction <- 
  stock_series_stats_div$market_cap__c____30_september_2020/sum(stock_series_stats_div$market_cap__c____30_september_2020)
stock_series_stats_div$real_estate_sub_sector <- NULL
stock_series_stats_div

p <- ggplot(data = filter(stock_series, real_estate_sub_sector == "Diversified")) + 
              geom_line(aes(x = as.Date(date), y = returns, group=stock_name_root, color=name, linetype=name)) +
                labs(title = "Category - Diversified",
                   x = "reference month", y = "monthly stock returns", 
                            group= "Ticker", color= "Ticker")
ggplotly(p, width=880, height=450)

2. Category: Specialized

The companies with a largest market capitalization exist also the longest. If we would weight according to capitalization, then only two companies would dominate an categorical indicator.

stock_series_stats_spec <- stock_series %>% 
  filter(real_estate_sub_sector=="Specialized") %>%
  group_by(stock_name_root) %>%
      summarize(
        min = min(date, na.rm=TRUE),
        max = max(date, na.rm=TRUE),
        real_estate_sub_sector = unique(real_estate_sub_sector),
        market_cap__c____30_september_2020 = 
          max(market_cap__c____30_september_2020),
        name = unique(name),
        .groups = "keep"
      )
stock_series_stats_spec$market_cap_fraction <- 
  stock_series_stats_spec$market_cap__c____30_september_2020/sum(stock_series_stats_spec$market_cap__c____30_september_2020)
stock_series_stats_spec$real_estate_sub_sector <- NULL
stock_series_stats_spec

p <- ggplot(data = filter(stock_series, real_estate_sub_sector == "Specialized")) + 
              geom_line(aes(x = as.Date(date), y = returns, group=stock_name_root, color=name, linetype=name)) +
                labs(title = "Category - Specialized",
                   x = "reference month", y = "close stock value", 
                            group= "Ticker", color= "Ticker")
ggplotly(p, width=880, height=450)

3. Category: Real Estate Development and Services

All 3 companies exist for a longer time at the stock market and have a similar market cap. One company would dominate a categorical indicator with 50%.

stock_series_stats_real <- stock_series %>% 
  filter(real_estate_sub_sector=="Real Estate Development and Services") %>%
  group_by(stock_name_root) %>%
      summarize(
        min = min(date, na.rm=TRUE),
        max = max(date, na.rm=TRUE),
        real_estate_sub_sector = unique(real_estate_sub_sector),
        market_cap__c____30_september_2020 = 
          max(market_cap__c____30_september_2020),
        name = unique(name),
        .groups = "keep"
      )
stock_series_stats_real$market_cap_fraction <- 
  stock_series_stats_real$market_cap__c____30_september_2020/sum(stock_series_stats_real$market_cap__c____30_september_2020)
stock_series_stats_real$real_estate_sub_sector <- NULL
stock_series_stats_real

p <- ggplot(data = filter(stock_series, real_estate_sub_sector == "Real Estate Development and Services")) + 
              geom_line(aes(x = as.Date(date), y = returns, group=stock_name_root, color=name, linetype=name)) +
                labs(title = "Category - Real Estate Development and Services",
                   x = "reference month", y = "close stock value", 
                            group= "Ticker", color= "Ticker")
ggplotly(p, width=780, height=450)

4. Category: Industrial/Office/Retail/Residential

stock_series_stats_ind <- stock_series %>% 
  filter(real_estate_sub_sector=="Industrial/Office/Retail/Residential") %>%
  group_by(stock_name_root) %>%
      summarize(
        min = min(date, na.rm=TRUE),
        max = max(date, na.rm=TRUE),
        real_estate_sub_sector = unique(real_estate_sub_sector),
        market_cap__c____30_september_2020 = 
          max(market_cap__c____30_september_2020),
        .groups = "keep"
      )
stock_series_stats_ind$market_cap_fraction <- 
  stock_series_stats_ind$market_cap__c____30_september_2020/sum(stock_series_stats_ind$market_cap__c____30_september_2020)
stock_series_stats_ind$real_estate_sub_sector <- NULL
stock_series_stats_ind

Top companies in company for market cap indicates that the market cap is fairly distributed and there may be a less dependency of a categorical indicator on individual companies.

head(stock_series_stats_ind %>% arrange(across(market_cap_fraction, desc)), 10)

p <- ggplot(data = filter(stock_series, real_estate_sub_sector == "Industrial/Office/Retail/Residential")) + 
              geom_line(aes(x = as.Date(date), y = returns, group=stock_name_root, color=name, linetype=name)) +
                labs(title = "Category - Industrial/Office/Retail/Residential",
                   x = "reference month", y = "close stock value", 
                            group= "Ticker", color= "Ticker")
ggplotly(p, width=880, height=450)

5. All Categories

stock_series_stats_all <- stock_series %>% 
  group_by(stock_name_root) %>%
      summarize(
        min = min(date, na.rm=TRUE),
        max = max(date, na.rm=TRUE),
        real_estate_sub_sector = unique(real_estate_sub_sector),
        market_cap__c____30_september_2020 = 
          max(market_cap__c____30_september_2020),
        .groups = "keep"
      )
stock_series_stats_all$market_cap_fraction <- 
  stock_series_stats_all$market_cap__c____30_september_2020/sum(stock_series_stats_all$market_cap__c____30_september_2020)
stock_series_stats_all$real_estate_sub_sector <- NULL
stock_series_stats_all

Fairly well distributed.

head(stock_series_stats_all %>% arrange(across(market_cap_fraction, desc)), 10)

p <- ggplot(data = stock_series) + 
              geom_line(aes(x = as.Date(date), y = returns, group=stock_name_root, alpha=0.2)) +
                labs(title = "All tickers",
                   x = "reference month", y = "close stock value", 
                            group= "Ticker", color= "Ticker")
ggplotly(p, width=880, height=450)